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Backg round of the Invention 

This invention relates generally to image and document image understanding, and 
more particularly to a system that can detect or recognize certain objects in a screening 
process. 

Screening for hazardous objects using a screener device is a very demanding task that 
requires both accuracy and efficiency. Human factors comprising sleepiness, fatigue, 
boredom, and inadequate training may affect the ability of a person to do this task accurately 
and efficiently. Unfortunately, this kind of failures may potentially lead to a disaster 

Upgrading the screener device may increase the overall performance. However, it is 
an expensive solution and does not guarantee that personnel with inadequate training or poor 
mental condition can do the task well enough. 

Although in near future nothing can substitute a state ofthe art screening device with 
a well-trained personnel in his or her tip-top shape, this system could potentially compensate 
some error made by less qualified device or personnel. To begin with, this system can be 
trained to recognize and mark potentially hazardous objects for further, more careful 
examination by the operator of the screening device. Moreover, the system can be interfaced 
with any TWAIN-compliant device. This means that with a suitable adaptor and driver, the 
system can be interfaced with the screening devices already being used. 



Summary of the invention 



The primary object of the invention is to recognize potentially hazardous objects 
during a screening process. 

Another object of the invention is to minimize screener's failure to recognize or to 
detect potentially hazardous objects during a screening process by recognizing and marking 
said objects automatically when they are displayed on a monitor. 

The system and method of this invention recognize objects trained by the user. Said 
system categorizes said objects into several classes, and marks said objects according to their 
classes. The system displays the representation of the recognized objects hierarchically. Each 
parent node displays a class of objects. Said user may expand said parent node to display the 
representation of said recognized objects that belong to that class. Once displayed, said user 
may choose the representation of an object to pinpoint the location and the class of said 
object. 

The system comprises an image processing subsystem, a recognition subsystem, and a 
training subsystem. 

The image processing subsystem acquires an image from a screening or image 
acquisition device such as an x-ray screening device by using standard TWAIN protocol. For 
a device without any compatible interfaces, a special adaptor that convert the available 
interface to a supported interfaces such as universal serial bus or parallel port along with an 
appropriate driver can be used. The image acquired from the device is processed further to 
increase the performance of the system. 

The object recognition subsystem uses the information acquired and processed by the 
hnage processing subsystem about the objects and their locations. The object recognition 
subsystem determines the boundary of each object in the image and recognizes them by using 
a pattern recognition engine tolerant to rotation and size. The object recognition subsystem 
recognizes each object in the image and categorizes each recognized object into object 
classes. 

The training subsystem is used to teach the object recognition subsystem to recognize 
new kinds of objects and re-leam old objects. 



The foregoing features and other aspects of this invention will now be described in 
accordance with the drawings in which: 

Figure. 1 is a diagram of the suggested application and requirement or configuration of the 
system to be used with a screening device. 

Figure.2 is a UML diagram of key elements in the system. 

Figure.3 is a diagram of the neural networks used to recognize pattern in the object 
recognition engine in the system. 

Detailed Descriptions of th e Preferred Embodiments 

Detailed descriptions of the preferred embodiment are provided herein. It is to be 
understood, however, that the present invention may be embodied in various forms. 
Therefore, specific details disclosed herein are not to be interpreted as limiting, but rather as a 
basis for the claims and as a representative basis for teaching one skilled in the art to employ 
the present invention in virtually any appropriately detailed system, structure, or manner. 

Referring now to Figure. 1 , the system is shown to comprise a screening device 1 . Said 
screening device 1 comprises a generic x-ray screening device. 

The system is shown to further comprise an adaptor 2. Said adaptor 2 converts video 
signal output from said screening device 1 to digital format. Said digital format follows 
standard and port that can be recognized by the system. 

The system is shown to further comprise a computer system 3, The computer system 
3 comprises personal computer that can run the software part of the system. Said computer 
system 3 displays data from said screening device 1 and pinpoints objects said computer 
system 3 recognizes as hazardous objects. 
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An operator 4 operates the system. Said operator 4 performs more thorough checking 
whenever the system detects possible hazardous objects. 

Referring now to Figure.2, the UML diagram of the system is shown to comprise 
TWAIN interface 20. Said TWAIN interface may control data acquisition from any TWAIN- 
compatible image acquisition device comprising a screening device 10. Said TWAIN 
interface then produces an image 30 of the actual objects being screened. 

The system is shown to fiirther comprise an image-processing subsystem 40, which 
comprises an image processing engine 4] and an object recognition engine 42. 

Said image-processing engine 41 receives said image 30 and applies image- 
processing techniques to enhance the quality of said image 30. Said image-processing 
techniques comprise dilation, image-depth conversion, and gray scaling. Said image- 
processing engine 41 converts said image 30 into several two-dimensional array image 
matrixes 43. Each image matrix 43 comprises a filtered version of said image. 

The object-segmentation engine 42 uses image matrix 43 to get the boundary of each 
object. The object-segmentation engine stores the information about said boundary of each 
object in a list of objects 44. 

The system is shown to fiirther comprise a recognition subsystem 50, which 
comprises an object recognition engine 51. 

The object recognition engine 5 1 receives said image matrix 43 and said list of objects 
44. The object recognition engine 51 retrieves the representation of each object in said image 
matrix 43 using data from said list of objects 44. The object recognition engine 51 produces 
object info 53 comprising the class and the hazard level of each object using a priority list 52. 
Said priority list 52 comprises a list of all classes of objects and their hazard levels. The 
object recognition engine 5 1 uses pattern recognition engine 54. Said pattern recognition 
engine 54 is a neural network pattern recognition engine tolerant to rotation and scaling. 



The system is shown to further comprise a user interface/object viewer 60. The user 
interface/object viewer 60 displays the class of each object recognized by said object 
recognition engine 5 1 hierarchically, grouped by their hazard levels, Said user 
interface/object viewer 60 pinpoints an associated object if a user chooses a class that 
represents that object. The way user interface/object viewer 60 pinpoints an object depends 
on the hazard level of that object. A monitor 70 displays the user interface/object viewer to 
said user. 

Referring now to Figured, the diagram of the artificial neural networks used to 
recognize pattern in the object recognition engine in the system shown to comprise input 
pattern 100. Said input pattern 100 is the pattern that will be recognized by the neural 
networks. Each pattern is a representation of an object the recognition system is trying to 
recognize. 

The neural network is shown to further comprise feature templates layer 110. Feature 
templates 1 10 are used to extract certain features from said input pattern 100. Feature 
templates 1 10 are arranged in several clusters, each cluster has the same number of templates. 

The neural network is shown to further comprise input neurons 120. Said input 
neurons 120 form an input layer. Each neuron in said input neurons 120 receives input from 
the result of feature extraction by a template in said feature templates 1 10 layer. Said input 
neurons are arranged in several clusters, each cluster has the same number of neurons. The 
number of neurons in each cluster is equivalent to the number of templates in a cluster in said 
feature templates 110. 

The neural network is shown to further comprise shift registers or ring buffers 1 30. 
Each shift register contains a certain number of elements. Each element receives input from a 
neuron in said input layer 120. The number of elements in each shift register is equivalent to 
the number of neurons in a cluster in said input layer 120. 

The neural network is shown to further comprise output neurons 140. Said output 
neurons 140 form an output layer. Many kinds of neural networks can be used in this layer, 
comprising variants of muttiplayer perceptrons (MLP) and variants of radial basis function 
(RBF) networks. This output layer receives input from said shift registers 130, 



While the invention has been described in connection with a preferred embodiment, it 
is not intended to limit the scope of the in vention to the particular form set forth. On the 
contrary, it is intended to cover such alternatives, modifications, and equivalents as may be 
included within the spirit and scope of the invention as defined by the appended claims. 



